D R a Ft on Architecture Selection , Cross - Validation , Andcomplexity

Determining the most appropriate network architecture for a data generating process (DGP) is a fundamental aspect of modeling relationships via artiicial neural networks. Cross-validatory techniques rank among the most popular approaches toward architecture-selection. Cross-validation is used to estimate the expected squared prediction error of a model. Architecture-selection via cross-validation proceeds from the fact that the true but unknown DGP will minimize expected squared prediction error, and selects that model which minimizes the cross-validatory estimate of expected squared prediction error. Conventional wisdom holds that cross-validation is an unbiased but highly variable method for determining the expected squared prediction error of a network architecture. This paper begins by demonstrating that the conventional wisdom may not hold in applied nonlinear settings, and then proceeds to an analysis of the bias and variance of network complexity which can often occur when network architecture is selected via cross-validation. It is shown how the use of standard algorithms such as conjugate gradient for the purpose of cross-validatory architecture-selection will tend to select unnecessarily complex network architectures. One of the main points made herein is that cross-validation can, in fact, be very reliable for the purposes of architecture-selection when care is taken in the training phase. Applications are considered.